Telling people to go back to where they came is definitely said by actual racist people, but the sentence 'why don't you go back to where you came from ...' is ambiguous.
Well, hell. . . ALL sentences are ambiguous!
The only way you can kind of disambiguate a sentence is by context. Even then, there is always some sort of ambiguity present in language. It's not a freaking code.
If I go out into the yard and tell my grandkids "Hey, why don't you all go eat some watermelon" and they go and eat some watermelon, that is certainly not racist.
If I were to use that sentence in other contexts, well. . . . .
SO yeah, it's a fact that sentences are ambiguous and we use context to disambiguate.
I am saying that in context, TO ME, not the media, not the democrats/liberals whatever, Trump's statement comes off as racially charged and motivated.